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Abstract — This paper provides a source coding theorem for 
multi-dimensional information signals when, at a given instant, 
the distribution associated with one arbitrary component of the 
signal to be compressed is not known and a side information is 
available at the destination. This new framework appears to be 
both of information-theoretical and game-theoretical interest : it 
provides a new type of constraints to compress an information 
source; it is useful for designing certain types of mediators in 
games and characterize utility regions for games with signals. 
Regarding the latter aspect, we apply the derived source coding 
theorem to the prisoner's dilemma and the battle of the sexes. 

I. Introduction 

Shannon has shown [10] that it is possible to describe with- 
out any loss a discrete random variable x G X (representing an 
information source) provided that H(x) bits per information 
source sample are used, H (x) being the entropy of the random 
variable x. This result holds for a stationary source that is, 
the distribution of x, say P(-) G A(A'), does not vary over 
time. In [4], [5|, Dobrusin introduced a generalized version of 
this problem where the source distribution can vary. In such a 
situation, the source distribution P(-\s) G A(A') can vary from 
sample to sample, depending on the parameter or state s G S. 
This problem, which is referred to as the problem of arbitrary 
varying sources, has been solved in Q. In [6| the author solve 
the problem of arbitrary varying sources when the sequence 
of states s n = (si, S2, s n ) is known to the destination (i.e., 
the decoder). The problem of arbitrarily varying source (AVS) 
is different to the one of universal source coding (USC), even 
if in both situations the encoder and decoder do not know the 
right distribution P(-) G A(A") of x. Indeed, the distribution 
probability of the symbols is constant for the USC problem 
but vary from stage to stage for the AVS problem. A further 
generalization of this work has been done in HI and 0. 
Indeed, the latter references deal with the scenario of two 
correlated sources either in the case where the destination is 
informed with the sequence of states or the case where it is not 
known. The work reported in this paper is precisely related to 
the scenario of two arbitrary varying correlated sources with 
an uninformed destination; this scenario is described by Fig. 
Q] For this scenario, which is motivated next, the achievable 
rate region in [1] is characterized for the family of information 
sources satisfying an entropy positiveness condition: 

Vs G S, H(x(s)\y{s)) x H(y(s)\x(s)) > 0, (1) 



where S is the set of possible states, x(s) and y(s) represent 
the two information sources to be encoded when the state is 
s G S. 

One of our contributions is to show that the condition ([TJ 
can be replaced with another mathematical condition which 
is fully relevant in dynamic games (at least in discrete-time 
dynamic games). The proposed condition is relevant in any 
game where an entity (a mediator typically) has to encode 
the action profiles of the game into messages intended for 
players having a certain observation structure. Considering 
that the relevant entity has to encode a sequence of action 
profiles, the proposed condition consists in assuming that at 
most one player can change his distribution in an arbitrary 
manner during the course of the game (single deviations) 
whereas the other do not change their strategy. One of the 
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Fig. 1. The pair of source symbols (x, y) £ X X y is drawn from the 
transition probability P(x.y|s) which depend on a state parameter s £ S. 
This parameter varies from stage to stages in an arbitrarily manner. The 
encoder C\ takes a sequence of symbols x" and encode it into a message 
mi. The encoder C2 takes a sequence of symbols y™ and encode it into a 
message 1112 . The decoder observes the pair of messages mi , 1112 and decode 
a pair of sequence of symbols (x" , y™ ) . 



motivations for deriving coding schemes which are resilient [7] 
to single deviations is therefore to consider the following class 
of relevant game-theoretic scenarios. This class consists of 
dynamic games with signals (the game is played several times 
and players do not observe the actions played perfectly) in 
which an exogenous entity (see Fig |2]l sends an additive signal 
so that players obtain better observations. More motivations for 
considering this type of scenarios can be found in J9)- 

II. Main technical result 

A. Definitions and closest existing result 

In this section, we consider an information source, rep- 
resented by Fig. [U where the symbols are drawn with a 
distribution which may vary from one stage to another in an 
arbitrary manner. Denote by S the set of states, X, y two sets 



of symbols and P(-\s) £ A(X x y); s £ S the distribution of 
the pair of random variables (x,y) when the state is s £ S; 
the set A(£) denotes the set or probability distributions on 
the generic set £ that is, the unit simplex associated with £ . 
For every sequence of states s n £ S n , define the probability 
distribution of the source sequence (x",y n ): 



Vs"eS n , P{x n ,y r ' 



l[P(x(t),y(t)\s(t)). (2) 



The arbitrary varying correlated source (AVCS) is defined by 
the conditional probability P(x, y\s) and by the set of possible 
sequences {s 11 } of source states. The component x (resp. y) 
of the correlated source is received by the first (resp. second) 
encoder C\ (resp. C2). Both encoders want to transmit their 
source component to the receiver. Our goal is to construct a 
joint code for which the error probability is upper-bounded in 
every state sequence s" £ S n . 

Definition 1: Define an (n, Mi, M2)-code as a triple of 
functions 
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X n — ► Mi, 

y n —>M 3 , 
Mi x M 2 — 



x n x y r - 



(3) 
(4) 
(5) 



where Mi = {1,2, ...,M<}, i £ {1,2}. With each code is 
associated an error criterion which has to be minimized. Define 
the error probability P™ associated to each (n, Mi, M2)-code 
as follows: 



max P((x",y")^(x J 



(6) 



Definition 2: A rate pair (i?i,i?2) is achievable if for all 
e > 0, there exists an (n, Mi, M2)-code such that : 
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(7) 

(8) 
(9) 



Denote by 1Z the set of achievable rates. 

Theorem 3 (Ahlswede /[7]?, /El?)." Assume that the arbitrary 
varying correlated source satisfies the entropy positiveness 
condition : 



Vs g 5, H{x{s)\y{s)) x H(y(s)\x(s)) > 0. 



(10) 



Then the achievable rate region 1Z equals the following rate 
region: 



Pi 
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sup H(x(s)\y(s)), 






p s eA(s) 
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sup H{y(s)\x(s)), 








Rl + i?2 
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sup ff(x(s),y(s)), 






p s eA(s) 



where the supremum is taken over the probability distributions 
P s e A(5). 

This characterization is not valid in general if we remove 
the entropy positiveness condition. In the following section, 



we consider a special but important class of information 
sources satisfying the orthogonal deviation condition presented 
in definition [5] We will see that this assumption allows us to 
get rid of the entropy positiveness condition. 

B. A new result 

We investigate a class of arbitrary varying sources as de- 
scribed in Fig. [2] for which the entropy positiveness condition 
([Tot is replaced with the orthogonal deviation condition (fT2l . 
The information source under investigation falls into the class 
of AVCS because the decoder observes a sequence y n of side 
information correlated with x n . Our main result concerns the 
source of vector of information x = (xi, . . . ,xk) £ X. 

Theorem 4: Let P k (-\s) £ A(X k ), k £ JC, s £ S the 
family of source distributions of Xfc, k £ JC and "I : A (A") — > 
A(y) the transition probability corresponding to the AVS 
described by Fig [2] The rate : 



1Z* = max 



max i/Xx-fely^ 



log Xfc 



(11) 



is achievable for the source x = (xi, . . . ,xk) and there is no 
lossless coding scheme providing a better rate. 
The proof of this theorem is not trivial and a sketch is provided 
in section [III] The purpose of what follows is to explain the 
notations used in the above theorem and its meaning. 



x« = (xy,_. . -,x^) m x 




Fig. 2. The information vector x = (Xk)keK & X = Y\keK ' s drawn 
with the family of probability distribution Pfc('|.s) G A(A" fc ), k £ K, s £ 
cS which depend on the state parameter s £ S. The decoder observes a 
side information y drawn from the transition probability ~I(y|x). The encoder 
C encodes a sequence of symbols x™ and send a message m of minimal 
cardinality M to the decoder T> such as reconstruct the sequence of symbols 
x n with an error probability arbitrarily small. 



Consider a family of sets (Xk)keic an d a family of dis- 
tributions Pj. £ A(Afc) over each component k £ fC. Denote 
X = Y[ kelc A(X k ) the product of sets; P £ A{X) the product 
probability defined by P = ® keK Pk and P_ fe £ A(X^ k ) 
the probability defined by P_fc = ,, k Pj where the fc's 
component has been removed. 

The information source x we consider is a vector x = 
(xi, . . . ,xk) of random variables. A sequence of state of the 
source s n £ S n is a sequence of probability distribution over 
one component j 6 K of the vector x" = (x™, . . . ,x^). 

Definition 5: An arbitrarily varying source of information 
satisfies the orthogonal deviation condition if the source of 



information is vectorial x = (xi, . . . ,xk) and the set of sates 
S n is defined by the equation dT2l . 



(12) 



Consider for example, the sequence s n = 
(Qj(l), . . . , Qj(n)) G S n of distributions over the component 
j € K. of the vector x n — (a:™, . . . , x^-) of information. At 
stage 1 < t < n the symbol Xj(t) is drawn according to 
Qj(t) G whereas the other symbols Xk{t) are drawn 

i.i.d. according to the probability distribution Pk G A(Xk). 
Hence, the vector of information x(t) is drawn, at stage 
t > 1, according to the following probability distribution: 



P(xi, . . . ,Xj, . . . , xr \s) = [Pi <8> ■ • ■ <8> Qj{t) ® 



Pk](x) 



We consider the situation where the decoder has a side 
information. It observes, through a transition probability ~1 : 
X — > A(y), a degraded version y of the source symbols 
x. The orthogonal deviation condition allow the transition 
probability ~1 to be deterministic. In such a case, the entropy 
positiveness condition is not satisfied. Then we conclude that 
conditions ( fTOb and (fT2l are disjoint. 

Definition 6: Define (n, M)-code as a triple of functions 



X n — > M, 

Mxy n — 



X r ' 



(13) 
(14) 



Each code is associated with an error criterion which aims at 
being minimized. Define the error probability P™ associated 
to each (n, M)-code as follows: 



P e " = max P((x",y")^(x",y")k")- 

s n 6S" 



(15) 



Definition 7: A rate R is achievable if for all e > 0, there 
exists a (n, M )-code such that: 



logAf 



P" 



< R 

< e. 



(16) 
(17) 



Denote TV the infimum of achievable rates R. 

For each component k G tC, define the associated graph Qk- 

Definition 8: The graph Qk — (Xk,Ek) is defined as 
follows. The source symbols x k G Xk are the vertex of the 
graph and there is an edges e = (xk,x' k ) G Ek between two 
vertices if and only if: 

3x_ k € Supp P_ k , 3y G y, 3d > 0, s.t. , 
m.m(~\(y\xk,x-k),~\(y\x' k ,x^k)) > S. 

Let C be a set of colors. A coloring of the graph Qk is a 
function <f>k ■ Xk — > C that satisfies: 

e = {x k ,x' k ) G E k , =^ 4>k(xk) ^ 4>k{x k )- 

Definition 9: The chromatic number Xk of the graph Q k is 
the cardinality of the minimal coloring of the graph Q k - 



Denote ~\ Xk : X- k — > ^(3^) the marginal transition 
probability of 1 over the set of signal y when fixing the 
symbol Xk G Xk- 



>x k '■ X-k 

X-k 



A(y), (is) 

l Xk (y\x- k ) =~\(y\x k ,x-k)- (19) 



Denote y^ the random signal drawn from the transition 1 Xk - 
The definitions of Xk and y are introduced to evaluate the 
optimal rate 1Z* defined by the equation (fTTT i. 

In the following sections, we provide a sketch of the proof 
of the theorem l4l (Sec. ITITb and we apply the derived results to 
two famous games (Sec. IIVb . Note that the theorem |4] holds 
for mixed strategies and when realizations of the game action 
profiles are i.i.d. The Shannon theory literature comprises tools 
(8) to extend the theorem [4] to more general setups (e.g., 
sources with memory or strategies with recall). 

III. SCKETCH OF PROOF OF THE THEOREM^] 

In this section, we present a sketch of the proof of theorem 
|4] which will be completely detailed in a separate paper. This 
section is divided in two pails called the achievability and the 
converse. 

A. Achievability 

The achievability consists in providing codes that achieves 
the rate 1Z* . We construct a coding scheme based on graph 
coloring and statistical tests. Two points should be considered 
carefully. First, it is possible that the sequence of side informa- 
tion y" provides relevant information to the decoder, for some 
sequence of state of the source. Second, the coding scheme of 
Slepian and Wolf [ill is n °t feasible. Indeed, the transition 
probability 1 generating the side information is controlled 
by the symbols x^ G X k of each component k G K, whose 
probability distribution Pk G A(Xk) is not necessarily known. 

The coding scheme. The encoder performs a statistical test to 
determine the component k G K, whose sequence of symbols 
Xk G Xk is the least typical for the distribution P k G A(Xk). 

The sequence of symbols x r k L G X k of the component k G 
K. will be encoded using the coloring of the corresponding 
graph Qk ■ The coloring property of definition [8] implies that 
the sequence of side information y n and the sequence of colors 
c n characterize a unique sequence of symbols x k G X k . 

Other components x" fe G X™ k will be encoded according to 
the transition probability ~1 and the sequence x k . We construct 
a partition (Xk,X k ) of the set of symbols X k - If the symbol 
Xfc G Xk is often used (partition Xk), the sequence of signals 
s™ fc , driven by the marginal probability ~[ Xk , is long enough to 
use a source coding with side information of Slepian and Wolf 
ifTTI . Otherwise (partition X k ), the information x_fc G X-k 
must be encoded directly, without any compression. The result 
of IfTTI states that the sequence of components x_fc G X-k can 
be recovered by the decoder with an arbitrarily small error 
probability. 

Evaluation of the rate of the code. For each e > 0, there 
exists a ni G N such that the probability of error of the coding 
scheme of Slepian and Wolf IfTTI is bounded by e > 0. Let 



n 2 > 



log|/C|+m|.y fc |log|A- fc | 



. We show that for all n > n 2 S N, 



the rate of the code is bounded by the following quantity: 



logM 



log \fC\ ■ xt ■ \X- k 



log n 



< 



hlogx fc H \og\X_ k \ 



H{x- k \y Xk ) + \ogXk 
log |/C| + 7»i|A3b| log |Af_ fc | , 



< max 

x k €X k 



(20) 



(21) 



< max 
fce/c 



2s 



max -ff(x_ fe |y ) + logXfc 



3e 



TZ* + 3e. 



(22) 



We provide a sequence of code such that the error proba- 
bility can be made arbitrarily small and in the same time the 
rate of the code is upper bounded by TZ* + 3e. 

B. Converse 

In the converse, we show that no code whose rate is less 
than TZ* is reliable (the error probability is lower bounded 
away from zero). To show the converse, we assume that the 
optimal rate TZ* is less than the following quantity: 



TZ* < max 



max H(x- k \y x 

x k £X k 



logXfc 



(23) 



Suppose the maximum in equation (TTTb is reached for the 
component k E K. and symbol x k E X k . The probability 
distribution of the vector of actions x = [x±, . . . , xk) is the 
product of marginal distributions of each component x k E X k 
with k £ K,. The optimum rate TZ* is then written as the sum 
of rates of the components TZ* — J^keK^-k- The converse is 
divided into two parts. 

First, assume that the encoding scheme has a rate TZ*. < 
logXfc. By the property of the minimum coloring, the same 
color c k can be generated by two different actions x k E X k 
and xu E X k . Then, the probability of error tends to 1 as n 
tends to infinity. 

Second, suppose that the sequence x k l is fully reconstructed 
by the player that decodes but the rate TZ*_ k satisfies: 

< H Sik (x- k \y Xk ) = max H(x^ k \y x J. (24) 

%k£X k 

In this case, the result of Slepian and Wolf ifTTI ensures that 
the probability of error tends to 1 when the sequence of states 
s k ~ x k and n tends to infinity. 



IV. Application to games 

The results provided so far can be applied to any discrete- 
time dynamic game an arbitrary observation structure. We 
consider the cases of the prisoner's dilemma and the battle of 
sexes. The information source to be compressed/encoded (by 
a mediator) is the action profile of the game. The destination 
(i.e., the decoder) is given by the set of players who observe 
the played actions through a certain observation structure. So 
far, we have been assuming that the communication links 
between the mediator and the players is perfect. Here we 
assume that each link from the mediator to a given player 
has a certain capacity denoted by Cq. 

The strategic form of the prisoner's dilemma is defined by : 



G = (fC, (Ak)keK, (ufc)fce/c) 



(25) 



where JC is the set of players \JC\ = 2, A k is their discrete 
action sets Ai = {T, B} and A2 = {L, R} whereas u k is the 
utility function of player k E K, such that : 



u k 



Ai x A 2 



kefC. 



(26) 



For the prisoner's dilemma, the utility function writes as the 
tale of Fig. [3] When player 1 plays B and player 2 plays L, 
the utility of player 1 is 4 and the one of player 2 is 0. 

L R 
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0,4 


4,0 


1,1 



Fig. 3. The utility matrix of the prisoner's dilemma. 

Suppose that the player play according to a probability 
distribution P E A(Ai) x A(„4 2 ) (mixed strategies). The 
problem is to evaluate the rate associated with the information 
source (generating the action profiles) knowing that one of the 
players is potentially deviating using a different probability 
distribution. Also suppose that the destination of the com- 
munication system has a degraded version s" of the source 
sequence a n from a transition probability: 



1:A 



A(5), 



(27) 



where A = A\ x A 2 - 

Numerical results. In order to illustrate the above result, 
we need to fix a transition probability 1 from the set of 
actions to a set of signals {si, s 2 , S3, S4}. The transition 
probability is defined as follows. When the action (T, L) is 
played, the signal of the decoder is drawn from the probability 
distribution described by Fig. [5] More precisely, the probability 
of observing the right signal s\ (which corresponds to the 
actions (T,L)) equals 1 — |e whereas the probability of 
observing another signal (s 2 , S3 or S4) is equal to \e. The 
parameter e E [0, 1] allows us to compare both situations: 



Jl 



\ s a» = (af,^) 




Fig. 4. Players Ji and J2 observe, through 1, a noisy version s n of the 
sequence of actions a™ = (a™, . . . , a^-). The encoder C transmits, through 
the channel ~lrj> the sequence of encoded actions a™ to the players J\ and 
J2. The players recover the sequence of past actions a n = a™ with an error 
probability arbitrarily small if and only if the rate TZ* of the source is less 
than the capacity Co of the channel 1q, fixed to 1.7 bit per second. 



when e = 0, the actions a = (01,02) G A are perfectly 
recovered and when e = 1, the signal do not provide any 
relevant information. When the action profile (T, R) (resp. 
(B,L) or (B,R)) is played the corresponding signal S2 (resp. 
S3 or S4) is also drawn with probability 1 — |e whereas the 
other signals are drawn with probability ie. 




1.5 2 2.5 
Utility of player 1 



Fig. 7. Achievable utility region for the prisoner's dilemma under capacity 
constraints. 



1 - ^ 

Si 


S2 


S3 


S4 



compression rate thanks to theorem [4] 



Fig. 5. The transition probability ~I of signals received by the decoder T> 
when the joint action (T, L) is played. 



Figure [6] represents the graphs Gi and Q2 over player's 
action which correspond to the above transition probability 
"I and the definition [8] of Sec. [II] 



T 



B 



L 



R 



Qi 



Q2 



Fig. 6. The graphs Q\ and Q2 of the players for e > 0. When e = 0, there 
is no edges in both graphs. 



As soon as e > the graph the players are described in 
Fig. [6] Note that if e > the coloring of both graphs is of 
cardinality of 2 and thus logxi = log%2 = 1. If e = the 
pair of graphs have no edges. 

We evaluate the rate of such a source in order to transmit the 
sequence of actions. However, the transmission of the source 
of information over a channel is not always feasible. A natural 
question arises and is illustrated by figure [4] Is it possible to 
transmit the source of action profiles through a channel with 
fixed capacity Co ? Recall that the source of information a with 
side information s can be transmitted reliably over a channel 
*"Io with capacity Co, if TZ* < Cq. We evaluate the optimal 



TV = max 



max ff(a_ fe |s 0fe ) + logx fc 



(28) 



Suppose that the capacity of channel ~[q equals Co = 1.7 
(in bit/symbol or bit/sample)and the parameter e = 0.5, we 
will see that only a subset of the feasible utility region can 
be achieved, for the communication system of Fig. |4] by an 
i.i.d. strategy (see e.g., Fig. [7] for the game of the prisoner's 
dilemma). For each pair of mixed actions P x G A(Ai) 
and P 2 G A(„4 2 )> we evaluate the rate TZ* of encoding 
the corresponding sequence of actions. If it satisfies the 
channel capacity condition TZ* < Co, the sequence of actions 
can be recovered even if one of the players deviates from 
Pk G A(Ak)- The Fig. [7] represents the region of utility 
vectors whose corresponding mixed actions satisfy the channel 
condition TZ* < Cq. 

We consider also the game of the "battle of sexes" whose 
utility matrices are represented in Fig. [9] For the channel 
capacity of Co = 1.7 bit/symbol and the same transition 
probability with precision parameter e = 0.5 (see Fig. |5j, we 
obtain the achievable utility region of Fig. [9] 

Remark that in both examples, we consider only mixed 
strategies. The problem of considering correlated strategies is 
much more complicated. Indeed, we need to define properly an 
arbitrary varying source where the actions and the deviations 
may be correlated. It would be also possible to achieve the 
same utility vector with various probability distributions. In 
that case, the optimal admissible rates may belong to an 
interval. Depending on the formulation of the problem, the 
capacity constraints may be defined with respect to the lowest 
or the largest optimal rate associated with a utility vector. In 
game theory, most of the player's strategy are not generated 



i.i.d. It should be of interest to extend our results to the case 
of a information source compatible with the game theoretical 
strategies using information spectrum methods JH. 

L R 
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Fig. 8. The utility matrix of the battle of sexes. 



V. Conclusion 

We have investigate the problem of encoding arbitrary vary- 
ing correlated sources also considered by Ahlswede in 1980 
Iffl . lHJ. The "entropy positiveness condition" ( TTOb is replaced 
by an "orthogonal deviation condition" (fT2l over the states of 
the source. Both conditions are disjoint because there exists 
some information sources that satisfies one condition without 
satisfying the other one. The main result characterizes the 
optimal rate of the source of information and a sketch of the 
proof is presented. We provide a game-theoretical application 
of our main result : a characterization of the rate of a source 
of player's actions in a long run game. Our coding scheme is 
resilient to any unilateral deviation of the player's strategies. 
In conclusion, the coding scheme cannot be manipulated along 
the duration of the game. The coloring techniques of Ahlswede 
are directly involved in the characterization of the optimal rate 
of an information source. 



2r 




Utility of player 1 



Fig. 9. Achievable utility region for the battle of sexes under capacity 
constraints. 
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